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non- science students (n=8) . Second, the authors applied an argumentation 
analysis model to the student papers. This allowed them to further identify 
differences in the argumentation structures of the papers. Based on the 
analyses, it was found that while all three interview populations were able 
to recognize differences between the two papers, their reasoning for such 
differences were rather ambiguous. Applying the argumentation analysis model 
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structure for the papers. This study draws on these findings to discuss ways 
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Abstract 

The purpose of this paper is to assess the differences in university oceanography 
students’ scientific writing. Specifically, we examine the argumentation structures of a 
high scoring paper and a low scoring paper. This study was conducted in an introductory 
level oceanography course in a large public university. In this course students use an 
interactive CD-ROM, "Our Dynamic Planet" (Prothero, 1995), which provides them 
access to geological databases. Students are instructed to use information from this CD- 
ROM to write the course required scientific technical paper. We considered the 
assessment of writing in two ways. First, we analyzed the differences among and within 
three populations' assessments of the students' written texts. The three populations had 
varying degrees of experience and knowledge in geological sciences and are as follows: 
instructors (n=4), science students (n=9), and non-science students (n=8). Second, we 
applied our argumentation analysis model to the student papers. This allowed us to 
further identify differences in the argumentation structures of the papers. Based on our 
analyses, we found that while all three interview populations were able to recognize 
differences between the two papers, their reasoning for such differences were rather 
ambiguous. Applying our argumentation analysis model allowed for further specification 
of differences in the argumentation structure for the student papers. We draw on these 
findings to discuss ways of teaching students the construction of argument in scientific 
writing. 



O 

ERIC 



3 



Assessment of differences (NARST 2001) 



3 



Assessment of differences in university oceanography students' scientific writing 

The purpose of this paper is to study the argumentation structure in student 
scientific writing from a disciplinary point of view. We are interested in providing a 
means for the writing genre of geological sciences to be readily accessible and available 
to students. This issue is examined in two ways. First, we consider how populations 
with varied knowledge and experience in geological sciences identify and describe their 
overall opinion of the two papers, authors use of evidence, authors use of figures, and 
conclusions made by the authors uses of evidence in students' scientific writing. Second, 
we propose an argumentation model to make visible differences among the student 
papers. 

Research on writing in professional communities has identified the importance of 
social practices and community norms in defining relevant rhetorical features required f 
or specific writing purposes within disciplinary contexts (Schwegler & Shamoon, 1991). 
One key feature in scientific writing is a community’s assessment and determination of 
what constitutes evidence in a given historical moment (Bazerman, 1988). The 
importance of evidence in science and the recognition of the defining role of disciplinary 
communities has led educational research examining writing to learn issues in science 
education to bring epistemological issues to the foreground (Keys, 1999; Prain & Hand, 
1999). 

Our study stems from an ongoing ethnographic study of discipline specific 
scientific writing in a university oceanography course (Kelly, Chen, Prothero, 2000; 
Kelly & Takao, 2000). In an earlier report of research from this course, the authors 
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(Kelly, Chen, Prothero, 2000) discussed how the epistemology of oceanography was 
constructed, portrayed, and construed through everyday processes of spoken and written 
discourse. Their analyses of the ways oceanography was framed through discourse 
processes revealed three issues regarding epistemology and science: (a) Talk about 
writing in science used situationally specific meanings to define scientific processes (e.g., 
definitions of what counts as an observation), (b) Classroom discourse processes 
positioned science as a way of knowing and communicated particular views about 
science (e.g., contextual, evidential, contingent), (c) Description of scientific writing 
processes identified ways science differs from other ways of communicating and 
knowing (Kelly, Chen, Prothero, 2000, p. 711-713). While this previous study focused 
primarily on how the instructors framed the discipline through instruction on scientific 
writing, in this paper we investigate the argumentation structure in students scientific 
writing. In this way, we examine the epistemological issues associated with formulating 
evidence in writing. 

Theoretical Framework 

The theoretical framework for this study builds on research of writing to learn 
science, argumentation in science, and the rhetoric of science more generally. Our 
review of recent literature in science education indicates a newly burgeoning interest in 
research on learning to write in science genres. To date, few research studies have 
focused on discipline-specific scientific writing, specifically pertaining to geological 
science. Our study aims to contribute to this relatively under-researched field (Bezzi, 
1999). 
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In a review of the research in the field of “writing to learn science,” Keys (1999) 
called for the use of scientific genres in instruction and the examination of classroom 
activities that encourage “integrated inquiry and writing” (p. 128). Writing of this sort 
suggests a tie between the shaping of written knowledge and epistemological issues 
related to the use of evidence. This issue has begun to be investigated by Prain and Hand 
(1999) who found that students were not able to explain how knowledge claims were 
established nor how “writing could act as an epistemological tool” (p. 160). The 
relationship of writing and knowledge production thus remains under-developed. 

The second research field informing our work is derived from studies of 
argumentation. These studies have analyzed students’ and teachers’ arguments to 
consider issues of student reasoning, engagement in scientific practices, and development 
of conceptual and epistemic understandings has begun to get renewed interest in 
education (Duschl, Ellenbogen, & Erduran, 1999; Jimenez- Aleixandre, Rodriguez, & 
Duschl, 2000; Kelly, Druker, & Chen, 1998; Kuhn, 1992; Newton, Driver, & Osborne, 
1999). Studies have examined evidentiary authority in teacher-student discourse (Russell, 
1983; Carlsen, 1997), ways students reason about socio-scientific issues (Patronis, Potari, 
& Spiliotopoulou, 1999), and the appropriation of scientific discourse in students’ small 
group conversations (Richmond & Striley, 1996). However, few studies have applied 
argumentation analysis to examine students’ use of evidence in writing, a central practice 
in scientific disciplines (Atkinson, 1999; Bazerman, 1988). 

The third part of our theoretical framework draws from studies in the rhetoric of 
science - the study of how scientists argue (persuade) in the making of knowledge 
(Harris, 1997, xii). The production of written texts has played a central role in scientific 
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communities and analysis of the history of the cultural practices associated with the 
production of such texts has identified how the uses and purposes of written knowledge 
have changed with changing mores in scientific communities (Atkinson, 1999; 
Bazerman, 1988; Swales, 1990). This work suggests that assessment of evidence is a 
cultural practice and can be investigated empirically. Textual analysis of students’ 
writing and the assessment of this writing by populations with varied experience and 
knowledge in geological sciences therefore represent ways of understanding how 
evidence is used in science, and how it can be used by students in their own writing - 
issues we consider in our empirical study. Our two-part study is designed to identify the 
role of disciplinary-specific knowledge in assessing evidence, and subsequently, to make 
visible the practices of formulating evidence in geology writing. Both aspects of the 
study are oriented toward the goal of making explicit to students the disciplinary 
practices associated with formulating written evidence. 

Educational Setting 

This study was conducted in an introductory level oceanography course in a large 
public university and is part of a larger on-going educational ethnography of the course 
over the past five years (Kelly, Chen, & Prothero, 2000; Kelly & Takao, 2000). We 
chose to study this course for its emphasis on scientific writing in geological sciences. 
Geological sciences has been identified as an underresearched field in science education 
(Bezzi, 1999) and there is a growing interest in the role of writing to promote learning in 
science (Keys, 1999). 
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This undergraduate course satisfies a university writing requirement and 
emphasizes scientific writing. There are 2 one-hour and fifteen minute lectures per week 
and an additional weekly two hour laboratory section for this course. Students in the 
oceanography course are provided with the use of an interactive CD-ROM to access 
geological databases. This CD-ROM, “Our Dynamic Planet” (Prothero, 1995), was 
created by the course professor. Data modules relating to plate tectonics are available 
through the still and moving graphics of this CD-ROM, such as earthquake locations and 
depths, volcanic locations, and the relative age of islands (Fig. 1). More information 
about the CD-ROM may be found at http://oceanography.geol.ucsb.edu/. 

Students use the information provided to them throughout the course via this CD- 
ROM, course lectures, laboratory sections, course textbook, and course reader to assist 
them in completing their mid-term scientific writing assignment. This assignment 
required students to write a technical paper characterizing three geographical areas using 
relevant geological data and to reconcile their findings with plate tectonic theory. The 
course reader provided an outline of the format for the technical paper and included 
descriptions and examples of each section of the paper. 

Research Methods 

Our study is organized into two parts. First, we considered the differences among 
and within three populations’ assessments of students’ scientific writing. This portion of 
the study sought to identify differences in interpretation of evidence by populations with 
differing subject matter knowledge. By comparing across the three populations 
(undergraduate students enrolled in oceanography, undergraduate students not enrolled in 
oceanography, course instructors), variations in the assessment of evidence could be 
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made apparent. Second, we applied an argumentation analysis model to evaluate the 
argumentation structure of students’ papers. We specifically focused one paper rated 
high by the course instructors and one rated low in order to make explicit the differences 
in uses of evidence. Through this two-part study we are interested in making the writing 
genre for geology accessible and available to students. 

The researchers (Takao and Kelly) and the course professor collaborated for the 
selection of the two papers used in this study (one rated high, one rated low in terms of 
overall quality). Although the two papers chosen for use in this study are not necessarily 
meant to represent every low scoring paper nor every high scoring paper, they were 
chosen to represent clearly different levels of overall quality. These two papers were 
coded so the authors’ identity and the papers’ scores were unknown to the interview 
participants with the exception of the professor. The high scoring paper was coded with a 
red sticker and the low scoring paper was coded with a blue sticker. During the 
interviews the two papers were simply referred to as "the blue paper" and "the red paper." 

The three populations in our study had varying degrees of knowledge in 
geological sciences: subject matter experts (n=4), the course professor and three teaching 
assistants; science students (n=9), university students taking the introductory 
oceanography course; and non-science students (n=8), university students who have not 
taken the introductory oceanography course. These populations were selected because 
we were interested in learning how the level of discipline specific knowledge would 
influence the assessment of the quality of arguments in the students’ papers. In other 
words, we wanted to gain an understanding of the implicit knowledge drawn upon by 
subject matter experts to assess the quality of geological writing - knowledge that may 
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not be available for the student population. In the first part of our study we attempt to 
elicit such knowledge. 

The participants were individually interviewed and asked to compare one low 
scoring student paper with one high scoring student paper in terms of participants’ overall 
opinion of the two papers, authors’ use of evidence, authors’ use of figures, and 
conclusions made by the authors. These four issues were identified as important factors 
to consider in scientific writing (Kelly, Chen, & Prothero, 2000). The participants were 
asked to read both papers and then given a copy of our open-ended interview protocol 
(Patton, 1990) to know what questions to expect during their interview (Table 1, 

Interview Protocol). The interviews lasted between 30 minutes to 90 minutes and with 
the consent of the participants each interview was audio and video taped. Each interview 
was transcribed word for word. The transcripts of the interview audio and videotapes 
served as data for our analysis across and within the three populations’ assessments of 
students’ scientific writing. 

In our initial analysis we open-coded (Emerson, Fretz, & Shaw, 1995) the 
transcripts and multiple categories emerged from the data. Further in-depth analysis led 
to synthesis of the multiple categories into three categories of interviewees’ positions 
regarding the four main issues: ambivalent, favorable, and unfavorable. We then 
reanalyzed the transcripts and focus-coded (Emerson, Fretz, & Shaw, 1995) the data into 
these categories. We produced a chart demonstrating the participants’ assessments of the 
low scoring and high scoring papers (Figure 2). As we describe in subsequent sections of 
this paper, analysis of the interviews revealed that the participants’ reasoning patterns 
were rather ambiguous regarding their assessments of the differences among these two 
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papers. This led us to make explicit the argumentation structures of the two papers, so 
that such differences could be made explicit to the student writers. 

In the second part of our study, we applied an argumentation analysis model 
(Figure 3) to examine the two papers used in the interviews. The argumentation analysis 
model used in this study was initially developed in a previous research study within the 
larger ethnography of this university oceanography course (Kelly & Takao, 2000). The 
epistemic level categories were based on Latour's (1987) model of scientific writing. 
Latour's analysis of scientific writing suggested that scientists typically try to move 
rhetorically from low induction facts (i.e., very specific, grounded claims) to more 
generalized statements (i.e., theoretical claims) with respect to specific constructs of the 
relevant disciplinary-specific knowledge. In developing our initial argumentation 
analysis model (Kelly & Takao, 2000) we considered geological sciences-specific 
knowledge in relation to the students' argument structures. In this model there were six 
epistemic levels, from the most specific, grounded claims (shown at the bottom of the 
model) to progressively more general, theoretical claims (shown at the top of the model). 
For the purposes of this study, we reviewed our initial model and in effort to make the 
categories more definitive we revised the epistemic levels into the following: 
representations of data, identification of topographical features, relational aspects of 
geological structures, data illustrations of geological theories or models, geological 
theory or model proposed by the author, description of geological processes and 
references to definitions, experts, and textbooks. In this study we added a seventh 
category to our epistemic levels which is labeled as "PC" (Personal Comment) because it 
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refers to statements in which the author is seemingly "commenting" to the reader. For 
complete definitions and examples of each epistemic level refer to Table 2. 

The process of sorting each proposition into the relative epistemic levels follows 
from our previous research study (Kelly & Takao, 2000). The assignment for writing the 
technical paper included dividing the paper into preset sections: abstract, introduction, 
observations, interpretations, conclusions, and figures (Kelly, Chen, & Prothero, 2000). 
We analyzed the differences in epistemic levels of propositions comprising the sections 
labeled "observations" and "interpretations" of the two student papers since that was 
where much of the inferential work was done. First, we typed the text from the 
observations and the interpretations sections of the two student papers into computer 
files. Then we labeled each sentence with a proposition number for future cross- 
reference. Next, we sorted each proposition into an epistemic level based on the 
definitions of the epistemic categories. Then we placed the respective proposition 
number onto a semantic network (Figure 3). The semantic network indicates whether the 
sentence was from either the students' observations section (circles) or interpretations 
section (squares). The initial placement of each statement into the respective epistemic 
levels was completed by analyst 1 (Takao) and checked by analyst 2 (Kelly). We 
collaboratively reviewed all cases of disagreement until a consensus was reached. We 
placed the most specific claims on the bottom of the model and began our numerical 
system from this level. The numerical system we used was designed for referencing 
purposes in our rhetorical analysis. Although the numbers are in ordinal progression, 
they do not represent a quantitative measure of generality, nor should they be considered 
a measure of validity. 
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To further analyze the argumentation structure we evaluated links among 
statements across and within epistemic levels. These links reveal the connections (lexical 
cohesions, see Halliday & Hasan, 1976) among students’ propositions and are depicted 
on the semantic network as lines connecting propositions. Figure 4 represents the links 
that we recognized among propositions within and across epistemic levels. 

We used the following five criteria to define the links: (a) Our first critierion was 
explicit links among statements across and within epistemic levels. We defined this as 
sentences using indexical phrases such as "this" or its" in relation to a preceding or 
following statement. To illustrate this point consider the following propositions, "The 
first area that is being examined is the Aleutian Island chain in the Northern Pacific 
Ocean" and "This area is just off the southwestern tip of Alaska (Fig. 1 area 1)" (high 
paper area 1 propositions 1 & 2, respectively). Proposition 2 uses "this" to explicitly 
reference the study area noted in proposition 1 and therefore they are linked to one 
another based on this first criterion, (b) Our second criterion was repetition of the same 
words or phrases within each area of the high and low scoring papers. We complied a list 
of words and phrases pertaining to geological content, such as “trench,” “magma,” 
“island,” “arc,” etc., based on the geological terms from the two papers (Table 3). The 
words and phrases from the list were searched for in the observations and interpretations 
sections of each area of the two papers by the following process: First, we typed each of 
the propositions from the observations and interpretations sections of the two students 
papers into Microsoft Excel files. Next, we used features of the Excel program to search 
the files for repetition of exact words by typing in the entire word, e.g. "area", or phrase, 
e.g. "abyssal plane," and doing a "find." In addition to the links list, we searched for the 
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same words or phrases specific to each area. For example, the low paper area 1 discusses 
an "abyssal plane" so we searched for that phrase throughout the observations and 
interpretations sections of that paper. This search was limited to this particular area 
because areas 2 & 3 of the low paper and areas 1 & 2 of the high paper did not reference 
an "abyssal plane." In our link table we did not show cases where words from the Links 
List were in only one proposition for an area since these propositions were not linked to 
other propositions, (c) Our third criterion was repetition of similar words or phrases 
regarding geological content. Again we used the Links List and the Excel files 
containing the students propositions and used the "find" feature of the Excel program. 

We typed in parts of a word to represent variations of a word, e.g. typed in "quake-" to 
represent “earthquake,” “earthquakes,” “quake,” “quakes,” or to represent various 
phrases, e.g. "Aleutian-" represented "Aleutians,” “Aleutian Island Chain.” Next, we 
manually reviewed each statement for similar words or phrases. For example, based on 
our analysis we linked the words "seismic" and "earthquake" which are geologically 
similar in nature (high paper area 2). The following in as example of similar phrases, the 
phrase "chain of islands" was linked to phrases consisting of "Aleutian-"(high paper area 
1) because for this area of study the author interchanges "chain of islands" with "Aleutian 
Island chain" and "Aleutians." For this category we also included words that had the 
same root word but had different suffixes, for example "ocean" and "oceanic." We point 
out that in our search for words in Table 3 that are hyphenated, for some study areas, we 
removed the hyphen and searched for the whole word. We did this in the areas that did 
not have variations of the hyphenated word which allowed for a more specific word 
search because a search for a hyphenated word is more general than a search for the 
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entire word itself. For instance, for the low paper area 1 we searched for "earthquake" 
instead of "quake-" whereas for the high paper area 1 we searched for "quake-" because 
the author interchanged "quake" with "earthquake." (d) Our fourth criterion was 
superordinate propositions. We linked propositions that consisted of subordinate terms to 
propositions with superordinate terms. For example, the superordinate terms "these three 
facts" from proposition 25 (high paper area 1) correlates with the subordinate three facts 
noted in propositions 19, 23, and 24. (e) Our fifth criterion was propositions that were 
sorted into more than one epistemic level. On the semantic network a link was used to 
connect the proposition numbers that are repeated across various epistemic levels. We 
point out that this repetition of proposition numbers on the semantic network could make 
the proposition density appear seemingly higher than the actual true number of 
propositions being represented on the chart. For example, for the high paper number area 
1 there are actually 24 propositions, 3 of which are sorted across three epistemic levels 
and 5 which are sorted across two epistemic levels, giving the appearance that there are 
35 propositions. However, a closer look at the chart will show that some of the 
proposition numbers are actually repeated and should not be mistaken to represent 
additional propositions. Based on these five criteria, we calculated the amount of links 
across propositions. 



Analyses & Findings 

We conducted our analyses process of the two student papers in two phases. 
First, we analyzed the interview transcripts across and within the three populations’ 
assessments of the high scoring and low scoring papers. Second, we applied our 
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argumentation analysis model to further illustrate the differences among the high scoring 
and low scoring paper. 

In the first phase of our analysis, we considered the data from the interview 
transcripts. The interviews focused on four main issues: participants’ overall opinion of 
the two papers, authors use of evidence, authors use of figures, and conclusions made by 
the authors. Several analyses lead to the synthesis of participants’ positions regarding 
these issues into three categories: ambivalent, favorable, and unfavorable. We defined 
these categories as follows. Ambivalent referred to participant responses that were neither 
favorable or unfavorable towards the issue. For example, the following quotes are from 
the transcripts regarding the participants’ overall opinion of the two papers. The first 
quote pertains to the high scoring paper and the second quote pertains to the low scoring 
paper and the coded data inside the parentheses reference the speaker, transcript page, 
and transcript line number, “not a great paper but it’s better written” (TAI 3;7;142), “it 
[low scoring paper] was alright I don’t think it got an A in my opinion” (OSI 9-l;l;012). 
Favorable referred to cases when the participant spoke positively of the issue under 
discussion. The following examples illustrate the participants favorable positions 
regarding their overall opinion of first the high scoring paper and second the low scoring 
paper, “excellent paper its easy to read its concise it doesn’t throw in a lot of irrelevant 
stuff its correct very excellent” (IE;20;480), “ it was really good I think that you’d get a 
good grade” (PSI 2;6;062). Unfavorable referred to participant responses that were 
negative regarding the issue at hand. For example, “it stinks” (PSI 2;4;042) describes this 
participants overall opinion of the high scoring paper, whereas “there’s a definite lack of 
understanding that’s apparent” (II; 13- 14;3 12) refers to the overall opinion of this 
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participant in regards to the low scoring paper. We point out that these categories do not 
necessarily imply that the participant favors one paper over the other with regards to each 
particular issue. For example, the participant could speak favorably of both papers in 
terms if the same issue. The following two quotes serve to illustrate this point and are 
from the same participant regarding their position of the authors’ conclusions for first the 
high scoring paper and second the low scoring paper: “conclusion well thought out” (OSI 
6-l;3;024), and “conclusion really good” (OSI 6-1 ;5;056). 

We created a chart demonstrating the participants’ positions in terms of the four 
main issues for the low scoring and high scoring papers (Figure 2). At this point in our 
analyses we examined the information in the chart and began to make comparisons 
between the low scoring and high scoring papers. Based on the chart, we found that more 
of the interview participants spoke favorably of the high scoring paper than of the low 
scoring paper in terms of the following issues: participants’ overall opinion of the paper 
(n=14 high paper, n=3 low paper), authors’ use of evidence (n=14 high paper, n=3 low 
paper), and conclusions made by the authors (n=12 high paper, n=6 low paper). In 
regards to the authors’ use of figures, the instructors (n=4) all spoke favorably of this 
category for the high scoring paper and unfavorably for the low scoring paper. On the 
other hand, more of both the science students and the non-science students spoke 
favorably of the use of figures in the low scoring paper (7 science students, 5 non-science 
students) than that of the high scoring paper (2 science students, 3 non-science students). 
This analysis revealed that in terms of these four issues, the high scoring paper is 
regarded by the three populations as favorable in more instances than that of the low 
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scoring paper. This was not a surprising result, but rather formed a basis to consider the 
reasons offered for the relative differences in uses of evidence. 

Comparison of the participants’ positions regarding the four categories indicated 
that in general the high scoring paper was favored relative to the low scoring paper by the 
three populations. Although it seems somewhat to be expected and logical that our study 
would find that a high scoring paper would be preferred rather than a low scoring paper, 
the reasons for which it was favored were nevertheless ambiguous. Analyses of the 
transcripts revealed that the participants’ were not particularly articulate in their 
reasoning for their relative positions regarding the four main issues. For example, a 
science student (OSI 3-1) states “the second one [high scoring paper] is a lot stronger 
than the first one [low scoring paper]... it’s definitely well written.” The transcripts 
following this statement indicated the participant lacked clear reasons for the preference 
for the high scoring paper as “stronger” and “well written.” This was a typical response; 
participants made general overall assessments of the papers, yet they were not specific in 
pointing out examples from the papers to support their positions. 

We recognize that perhaps this lack in discussion of specific reasons for 
preferring one writing sample over another was due in part to the interviews themselves 
(Kvale, 1996; Mishler, 1986; Spradley, 1979). However, even the geology graduate 
student graders were ambiguous about the differences between the high scoring and low 
scoring papers. Therefore, to unpack such the differences in uses of evidence across the 
two papers, we applied our argumentation model to the students' writing. 

In the second phase of our analysis, we applied our argumentation analysis model 
to the high scoring and low scoring papers and noticed several differences among them. 




18 



Assessment of differences (NARST 2001) 



18 



First, in the high scoring paper, theoretical claims (epistemic level V) were supported 
with data presented across multiple epistemic levels (I, n, n, IV). To illustrate this point 
we now present the argumentation structure for the high paper Area 1: Aleutian Islands. 
The author, "Steve," used earthquake, volcanic, and topographical (i.e., trench) evidence 
to support his theoretical claim that a convergent subducting plate boundary exists at the 
Aleutian Island chain. These three sub-arguments were constructed with claims across 
various epistemic levels. For example, consider the earthquake evidence he used for that 
particular argument. Steve first introduced the subject of earthquakes in proposition 4 
which was sorted into epistemic level PC, "The first thing that I looked at is the 
earthquake activity in the area." " He referenced several data representations (epistemic 
level I) to provide support for his theoretical claim, such as in proposition 14: "Figures 8 
and 9 show the depth of the earthquakes along the profiles that were plotted." He built 
his argument by identifying topographic features in propositions that were sorted into 
epistemic level II such as, "There appears to be at least two hundred earthquakes directly 
along the chain of islands (Fig. 2)" (proposition 5). The next level of generality pertains 
to relational aspects of geological structures (epistemic level III) and the following 
proposition (#15) was sorted into that category, "As observed in figure 8, the majority of 
the quakes are within 50 km of the surface, however there are about 20 recorded 
earthquakes that extend down the trench to around -252 km." Steve's argument is further 
supported with proposition 19 which illustrated his theoretical claim with geological data 
from the study area (epistemic level IV), "The abundance of earthquakes in the area and 
their locations is the first clue as to what type of tectonic process we are seeing." For this 
particular argument, Steve also included a proposition which references information from 
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the course textbook (epistemic level VI), "It is known that at convergent plate boundaries 
there are deep earthquakes, volcanoes, and a deep trench at the boundary between the two 
plates (Ross, Oceanography, p. 52)". These propositions (epistemic levels I, II, HI, IV, 
and VI) served as evidence for his theoretical assertions in propositions 26 and 25 
claiming the study area is a subduction zone (Figure 5). 

Steve continued to use evidence across epistemic levels to support his theoretical 
claims in the second study area of his paper. Again he used earthquake evidence as noted 
in proposition 2, "The first step to determining any boundary type is by looking at the 
earthquake activity in the area" (epistemic level PC). He used several data references in 
propositions 5 and 18 (epistemic level I) and built his argument by identifying 
topographic features of the area (epistemic level II) such as in proposition 3, "Along this 
stretch of the ridge, about 3500 km in length, there are about 100 seismic occurrences 
(Fig. 3)." Steve referenced data (epistemic level IV) in proposition 18 to further support 
his theoretical claim (epistemic level V) that the Mid Atlantic Ridge is a plate boundary 
(propositions 12 and 22). 

In comparison, the low scoring paper had multiple general geological theories 
which were left relatively unsupported by specific reference to and use of geological data 
throughout the three study areas. We now turn to the argumentation structure of the low 
paper, specifically the first geographical area identified by the student (South America) to 
illustrate this point. The author, "Linda 1 ," made the theoretical claim (epistemic level V) 
that "Many factors indicate that the Western Coast of South America is a subduction 



1 The two papers were chosen without the reviewers knowing the students’ gender. As it turned out, the 
strong paper happened to be written by a male student, and the weak paper by a female. However, in a 
larger, representative there were no gender differences as measured argumentation strength or in student 
grades (see Kelly & Takao, 2000). 
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zone" in proposition 14. Linda used earthquake evidence as one of these factors. Unlike 
the high paper, in which case we could trace Steve's use of earthquake evidence across 
epistemic levels, in this low paper the evidence exists in only one epistemic level from 
one proposition, which is proposition 17: "The earthquakes along the trench also indicate 
a subduction zone." When analyzing this paper we found it difficult to decipher the other 
"factors" she intended as evidence. Therefore, this particular area illustrated a case in 
which Linda's theoretical claims were seemingly unsupported (Figure 6). In the case of 
area 2, again Linda presented earthquake evidence to support her theoretical claim 
(epistemic level V) that "The Himalayas were formed by a continental/continental plate 
collision (See Figure 4)" (proposition 7). Earthquake evidence was referenced in 
propositions 4 and 5 which were sorted across only epistemic levels II and III leaving her 
theoretical assertions relatively unsupported. For area 3, Linda discussed volcanic 
activity as evidence for her theoretical assertion (epistemic level V) that "The Hawaiian 
Island Chain was formed by a hot spot" (proposition 1 1) in only one proposition (6) 
(epistemic level II). She did reference other volcanic information in propositions that 
were sorted into epistemic level V (propositions 1, 3, 7, 10, and 16) and VI (propositions 
12, 14, and 15) which essentially do not use data from the study area as evidence to 
support her theoretical claims . In sum, Linda made a large number of unsupported 
theoretical assertions. 

Second, propositions from the high scoring paper were relatively evenly 
distributed across the various epistemic levels in comparison with that of the low scoring 
paper. A majority of the propositions from the low scoring paper were classified into 
epistemic level V or higher (Area 1: 17 of 28; Area 2: 11 of 17; Area 3: 18 of 24) (Figure 
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7). The distribution of propositions is related to our first finding discussed above. As 
previously noted, the high paper used evidence across multiple epistemic levels so that 
we would see these propositions distributed across our semantic network. On the other 
hand, the low scoring paper has a more skewed distribution of propositions which relates 
to our previous finding that supporting evidence was generally absent as there were few 
lower-inference propositions aligned with particular theoretical assertions. In addition, 
theoretical assertions of epistemic level V in the high scoring paper were proposed 
strictly in their "interpretations" section, whereas the low scoring paper had theoretical 
statements from both "observations" and "interpretations" sections. This may indicate 
that Linda was unclear about the level of inference permitted for “observations.” 

Third, when analyzing the links across propositions (criteria a-d) we noticed that 
the high paper has a significant amount of more links for each area, respectively, in 
comparison to the low paper. For the high paper area 1 there are 245 links and for area 2 
there are 222 links. However, for the low paper area 1 there are 129 links, area 2 has 48 
links, and area 3 has 154 links. We recognize that each area of each paper have a 
different amount of propositions which could account for some of the variation in amount 
of links that we see. In terms of the high paper, area 1 has 29 propositions and area 2 has 
22 propositions. In regards to the low paper, area 1 has 28 propositions, area 2 has 17 
propositions, and area 3 has 24 propositions. Nonetheless, it is interesting to note the 
difference in the amount of links among the high and low papers and the difference in the 
density (number of links / number of propositions) of the links for each area (Table 4). 

Although it may seem obvious that links between statements are key components 
to building a strong argument, this point was not articulated during the interviews. For 
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instance, findings based on the interviews did not indicate that a difference among the 
two papers was that the theoretical claims of the high paper was linked to supporting 
evidence, but we do see this from our model. In other words, our model did allow us to 
draw out some differences among the papers which were not articulated in the interviews, 
even for geology graduate student graders. Our findings ultimately allowed us to identify 
key components of the argumentation structures valued by assessors of evidence, even if 
such structure was not readily recognizable to them as such. 

Discussion & Educational Implications 

Based on our findings we point to the differences among the high scoring paper 
and the low scoring paper and suggest components to consider when constructing 
arguments in scientific writing. Our argumentation analysis model we proposed in this 
study provides a means to sort out differences in students’ scientific writing. This model 
helped to make explicit distinctions between university students’ high scoring and low 
scoring papers that were not otherwise articulated during the interviews. We recognize 
that the two papers in our study are unique in their own right, and undoubtedly have 
idiosyncratic features. Nevertheless, by applying the model to the students’ papers we 
recognized differences among them and made several inferences in terms of writing 
strong scientific arguments. 

The argumentation structure of strong scientific arguments includes theoretical 
claims that are supported by evidence across various epistemic levels. The distribution of 
propositions in strong arguments is relatively evenly distributed across the epistemic 
levels and the propositions should be linked to one another in building cohesive 
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arguments. These components can serve as a framework for students to use when 
constructing written scientific arguments. In addition, the assessment of student writing 
may be made more consistent by making the argumentation structures in scientific 
writing explicit. Examples of these applications can be found in the Instructor Manual 
for the CD-ROM "Our Dynamic Planet" (Prothero & Kelly, 2000). 

Specific implications can be drawn for instruction in writing in this genre, for 
both students and teachers. The task of formulating scientific arguments requires 
abstraction from specific data to make theoretical claims. However, this abstraction 
process requires intermediate steps, where the level of claim is more than description of 
data representations, but not as general as the overall theoretical assertion of the main 
argument. This requires student understanding certain features of scientific writing. First, 
students may need experience assessing the level of claim in their own and others' 
writing. Experience assessing the extent of the generality of claims may give students 
insight into the types and extent of evidence required for a given claim. Second, the facts 
need to be "stacked" from most descriptive to most general through progressive 
abstraction (Latour, 1987). Building a cohesive argument requires making connections 
across the levels of abstraction. Assessing the relationship of different parts of an 
argument, and how evidence is formulated for each part, may give students better insight 
into ways of unpacking the presented evidence. This may have significant implications 
for science-technology-society issues, where there are often ethical implications mixed 
with scientific information. Finally, for teachers of scientific writing, the use of this 
argumentation model identified previously invisible features of the writing task. 
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Considering the ways that claims are made in an explicit manner may give instructors 
insight into their own tacit knowledge embedded in the writing process. 
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Table 1: Interview protocol 



Questions: 

1) What do you see as the central purposes of the paper? Why do you think so, please 
explain. 

2) Can you identify the main conclusions of the author? Please explain how you arrived 
at this interpretation. 

3) What evidence has the author presented for her/his conclusions? Can you point to 
specific examples? 

4) Can you find examples of appropriate uses of evidence, that is, a case where you feel 
the conclusions is justified with data through a sound argument? 

4a) How did the author use the Figures or diagrams to support the conclusion of 
the paper? 

4b) Would you identify one Figure or diagram and discuss how the author uses it 
in her/his paper? 



5) Can you find examples of where the evidence is lacking or insufficient for the 
conclusions made? 

5a) Find a particular conclusion and try to identify the reasoning behind it. 

6) How would you compare the two papers? Which one was better written? Why do you 
think so? 

7) Do you have any questions that you would like me to answer? 
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Table 3: Proposition links words and phrases 



1. Age 

2. Area 

3. Boundary 

4. Colli- 

5. Continent- 

6. Converg- 

7. Crust 

8. Deep- 

9. Depth 

10. Erosion 

1 1 . Hot spot 

12. Island arc 

13. Island 

14. Land 

15. Magma 

16. Mantle 

17. Mountain 

18. Ocean- 

19. Plate 

20. Profile 

21. Quake- 

22. Ridge 

23. Rock 

24. Seafloor 

25. Sediment 

26. Seismic 

27. Spreading 

28. Subduct- 

29. Surface 

30. Topograph- 

31. Trench 

32. Volcan- 

33. Weathering 



**Plus words/phrases specific to area of study 
Ex. Low Paper Area 1 : Abyssal Plane 

Turbid- current 
South America 

Low Paper Area 2: Himalayas 

Low Paper Area 3: Hawaiian Island- 



High paper Area 1: 
High paper Area 2: 



Aleutian island chain 
Sea floor spreading 
Age of sea floor 
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Table 4: Density of links per propositions 





# Links 


High Scoring Paper 
Area 1 


245 


High Scoring Paper 
Area 2 


222 


Low Scoring Paper 
Area 1 


129 


Low Scoring Paper 
Area 2 


48 


Low Scoring Paper 
Area 3 


154 



# Propositions 


# Links / # Propositions 


29 


8.45 


22 


10.09 


28 


4.61 


17 


2.82 


24 


6.42 
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Figure 1: Map and earthquake depth profile from CD-ROM "Our Dynamic Planet" 
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